Simulation of an Erodible Tip in a Natural Media Drawing and/or Painting Simulation

ABSTRACT

A method, system, and computer-readable storage medium are disclosed for simulation of an erodible tip. A brush tool representing an erodible media is modeled as a height map. Information is collected about a user manipulation of a stylus representing a stroke made on a virtual canvas with the brush tool. A mark to be made on the virtual canvas is determined dependent on the brush tool model and the collected information. The determined mark is rendered. A change in the height map of the brush tool due to the stroke is determined dependent on the brush tool model and the collected information. One or more subsequent marks are rendered in response to manipulation of the brush tool dependent on the determined change in the height map.

PRIORITY INFORMATION

This application claims benefit of priority of U.S. ProvisionalApplication Ser. No. 61/606,204 entitled “Methods and Apparatus forSimulation of an Erodible Tip in a Virtual Pastel Simulation” filed Mar.2, 2012, the content of which is incorporated by reference herein in itsentirety.

BACKGROUND

1. Field of the Invention

The present invention is directed to computer systems; and moreparticularly, it is directed to the processing of digital images.

2. Description of the Related Art

Digital images may include raster graphics, vector graphics, or acombination thereof. Raster graphics data (also referred to herein asbitmaps) may be stored and manipulated as a grid of individual pictureelements called pixels. A bitmap may be characterized by its width andheight in pixels and also by the number of bits per pixel. Commonly, acolor bitmap defined in the RGB (red, green blue) color space maycomprise between one and eight bits per pixel for each of the red,green, and blue channels. An alpha channel may be used to storeadditional data such as per-pixel transparency values. Vector graphicsdata may be stored and manipulated as one or more geometric objectsbuilt with geometric primitives. The geometric primitives (e.g., points,lines, polygons, Bézier curves, and text characters) may be based uponmathematical equations to represent parts of digital images.

Digital image processing is the process of analyzing and/or modifyingdigital images using a computing device, e.g., a computer system. Usingspecialized software programs, digital images may be manipulated andtransformed in a variety of ways. For example, digital images may becreated and/or modified using natural media drawing and/or paintingsimulation. Natural media drawing and/or painting simulation refers todigital, computer-based creation techniques for creating digitaldrawings, digital paintings, or other digital works that attempt tomimic real-world techniques and results for drawing (e.g., using apencil and canvas) and/or painting (e.g., using a brush, palette, andcanvas).

The characteristic behaviors of real-world artist tools such as graphitepencils or oil pastels include interaction with the canvas texture, thedirtying of the tip, and the change in shape of the tip over time due toerosion. For example, during a stroke of a pencil on a canvas, a mark ismade by pigment breaking off the end of the pencil graphite and becomingembedded in the canvas. Therefore, the consumption of the pigment mediumis a fundamental aspect of how the tool behaves. By shaping the tipwhile stroking and then adjusting the tool pose to take advantage of thenew shape, talented artists are able to use this effect to theiradvantage. For example, an artist might make many broad strokes with apencil to create a chisel tip and then hold the pencil vertically tomake a thin line with the resulting chisel edge. Although some existingdigital painting applications support canvas texture and tip dirtyingeffects, they do not support tip erosion.

SUMMARY

Various embodiments of methods and apparatus for simulation of anerodible tip in a natural media drawing and/or painting simulation aredescribed. In some embodiments, a graphics application or image editingapplication (e.g., a natural media painting application) may simulatethe shape-changing property of a pastel, charcoal, crayon, or pencil, orany other type of mark making tool that has an erodible tip. Whencombined with canvas texture and mixer brush technologies, this maycreate a complete simulation of the behavior of an oil pastel, forexample.

In some embodiments, a brush tool representing an erodible media ismodeled as a height map and an associated triangle mesh attached to abrush handle. Information may then be collected about a usermanipulation of a stylus representing a stroke made on a virtual canvaswith the brush tool. A mark to be made on the virtual canvas may bedetermined dependent on the brush tool model and the collectedinformation. The determined mark may then rendered. A change in theheight map of the brush tool due to the stroke may be determineddependent on the brush tool model and the collected information. Datarepresenting an image that includes the rendered mark may be stored. Oneor more subsequent marks may be rendered in response to manipulation ofthe brush tool dependent on the determined change in the height map.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating one embodiment of a computerdevice that implements a natural media drawing and painting applicationemploying a tablet and a stylus device, as described herein.

FIG. 2 is a block diagram illustrating a display on which a userinterface to a graphics application may be implemented, according tosome embodiments.

FIG. 3 is a flow diagram illustrating one embodiment of a method forcreating a mark in a natural media painting application using a virtualbrush tool modeled with an erodible tip.

FIG. 4 illustrates various components of an example stylus, according tosome embodiments.

FIGS. 5A-5D illustrate various stylus poses and gestures that may berecognized by an interface module of a natural media paintingapplication, according to various embodiments.

FIGS. 6A-6B illustrate examples of various stylus poses during strokesmade on a tablet, according to some embodiments.

FIGS. 7A-7J illustrate various embodiments of a virtual brush with anerodible tip model.

FIGS. 8A-8C illustrate stamp generation based on an erodible tip contactarea, according to some embodiments.

FIG. 9 is a flow diagram illustrating one embodiment of a method forstamp generation a natural media painting application.

FIGS. 10A-10B illustrate the erosion of an erodible tip during a stroke,according to some embodiments.

FIG. 11 is a flow diagram illustrating the use of an erodible tip in anatural media painting application, according to some embodiments.

FIGS. 12A-12D illustrate stylus tilt scaling, according to oneembodiment.

FIG. 13 is a flow diagram illustrating one embodiment of a method forscaling the tilt of a stylus in natural media painting application.

FIG. 14 illustrates various components of tablet input device, accordingto some embodiments.

FIG. 15 is a block diagram illustrating an example computer system thatmay be used to implement the techniques described herein, according tosome embodiments.

While various embodiments are described herein by way of example forseveral embodiments and illustrative drawings, those skilled in the artwill recognize that embodiments are not limited to the embodiments ordrawings described. It should be understood, that the drawings anddetailed description thereto are not intended to limit the embodimentsto the particular form disclosed, but on the contrary, the intention isto cover all modifications, equivalents and alternatives falling withinthe spirit and scope of the disclosure. The headings used herein are fororganizational purposes only and are not meant to be used to limit thescope of the description. As used throughout this application, the word“may” is used in a permissive sense (i.e., meaning having the potentialto), rather than the mandatory sense (i.e., meaning must). Similarly,the words “include,” “including,” and “includes” mean including, but notlimited to.

DETAILED DESCRIPTION OF EMBODIMENTS

In the following detailed description, numerous specific details are setforth to provide a thorough understanding of claimed subject matter.However, it will be understood by those skilled in the art that claimedsubject matter may be practiced without these specific details. In otherinstances, methods, apparatuses or systems that would be known by one ofordinary skill have not been described in detail so as not to obscureclaimed subject matter.

Some portions of the detailed description which follow are presented interms of algorithms or symbolic representations of operations on binarydigital signals stored within a memory of a specific apparatus orspecial purpose computing device or platform. In the context of thisparticular specification, the term specific apparatus or the likeincludes a general purpose computer once it is programmed to performparticular functions pursuant to instructions from program software.Algorithmic descriptions or symbolic representations are examples oftechniques used by those of ordinary skill in the signal processing orrelated arts to convey the substance of their work to others skilled inthe art. An algorithm is here, and is generally, considered to be aself-consistent sequence of operations or similar signal processingleading to a desired result. In this context, operations or processinginvolve physical manipulation of physical quantities. Typically,although not necessarily, such quantities may take the form ofelectrical or magnetic signals capable of being stored, transferred,combined, compared or otherwise manipulated. It has proven convenient attimes, principally for reasons of common usage, to refer to such signalsas bits, data, values, elements, symbols, characters, terms, numbers,numerals or the like. It should be understood, however, that all ofthese or similar terms are to be associated with appropriate physicalquantities and are merely convenient labels. Unless specifically statedotherwise, as apparent from the following discussion, it is appreciatedthat throughout this specification discussions utilizing terms such as“processing,” “computing,” “calculating,” “determining” or the likerefer to actions or processes of a specific apparatus, such as a specialpurpose computer or a similar special purpose electronic computingdevice. In the context of this specification, therefore, a specialpurpose computer or a similar special purpose electronic computingdevice is capable of manipulating or transforming signals, typicallyrepresented as physical electronic or magnetic quantities withinmemories, registers, or other information storage devices, transmissiondevices, or display devices of the special purpose computer or similarspecial purpose electronic computing device.

Various embodiments of methods and apparatus for simulation of anerodible tip (e.g., in a natural media drawing or painting simulationsuch as a virtual pastel simulation) are described. Various embodimentsmay provide methods for performing various digital painting and drawingtasks using a natural, posed-based and/or gesture-based approach via atablet, stylus, and software such as the tablet/stylus input module ofthe graphics application described herein. Some embodiments may providedetection of stylus gestures that mimic the real-world actions ofartists in real (as opposed to digital) painting and drawing. Someembodiments may provide detection of stylus poses that mimic the way inwhich real-world artists create different effects by manipulating thepose (e.g., the orientation, position, and/or tilt) of a brush, pencil,oil pastel, charcoal, crayon, airbrush, or other natural media vehicle.Some embodiments may perform appropriate digital painting and drawingactions and/or produce appropriate painting and drawing effects inresponse to detecting the stylus poses and/or gestures. In someembodiments, the erodible tip simulation may lower the cognitive load ofthe user and allow the user to focus on creativity as opposed to theworkings of conventional graphics applications and the user interfacesprovided therein. Various embodiments may enable gesture-based naturalmedia painting workflows by providing a set of six degrees of freedom(6DOF) stylus-based gestures and poses for use with a stylus and tabletinput technologies, which may be augmented with additional hardware, andby mapping these stylus gestures and poses to painting tasks and effectsin a natural media model.

Using a stylus, tablet, and software such as a tablet/stylus inputmodule of a graphics application, such as that illustrated in FIG. 1,some embodiments may collect data from the stylus and/or tablet inresponse to user manipulation of the stylus and/or tablet to performvarious user manipulation tracking tasks. The collected data mayinclude, but is not limited to, acceleration, position, orientation, andproximity data detected for or by the stylus, and touch and pressuredata detected for or by the tablet. The collected data may be used torecognize various stylus poses and/or gestures in real-time or near-realtime, and the recognized stylus poses and/or gestures may be mapped toappropriate real-world painting and drawing actions that are thensimulated in the graphics application as described below.

FIG. 1 illustrates an example graphics workstation or other computingdevice that is configured to implement the systems and methods describedherein, according to various embodiments. As illustrated in thisexample, the workstation may include, but is not limited to, a computerdevice 100, one or more displays 102, a tablet input device 130, and astylus 140. An example computer device which may be used in someembodiments is further illustrated in FIG. 15. As illustrated in thisexample, computer device 100 may implement a graphics application 120,which may be a natural media painting application, as described herein.Graphics application 120 may include a brush model 123, a paintingsimulation module 127, a tablet/stylus input module 122, and/or apose/gesture mapping module 125. In some embodiments, brush model 123may be a component of painting simulation module 127.

Graphics application 120 may provide a user interface (UI) 124 which maybe presented to a user via one or more displays 102. Graphicsapplication 120 may display, for example in a window provided by the UI124 on the one or more displays 102, an image 126 that a user iscurrently working on (e.g., either creating or editing). Graphicsapplication 120 may provide a painting or drawing tool 128 that the usermay manipulate, for example via tablet 130 and/or stylus 140, to createor edit content in image 126. The tool 128 may, for example, havevarious modes that emulate a paintbrush, pencil, charcoal, crayon, oilpastel, eraser, airbrush, spray can, and so on. While embodiments aregenerally described as providing gesture-based manipulations of apaintbrush tool, it is to be noted that similar techniques may beapplied to other types of painting or drawing tools.

Stylus 140 may be configured to be held in a hand of the user and to bemanipulated by the user in relation to tablet 130 to perform variousimage editing operations or other tasks. The user may manipulate stylus140 and/or tablet 130 in various ways. For example the user may movestylus 140 away from tablet 130 or towards tablet 130; move stylus 140up and down, left and right, or diagonally and so on in relation totablet 130; rotate stylus 140 on one or more axes; touch a touch and/orpressure sensitive surface of tablet 130 with stylus 140 and/or with afinger, knuckle, fingernail, etc.; apply varying amounts of pressure tothe touch and pressure sensitive surface of tablet 130 with a finger orstylus 140; move the tip of stylus 140 on the touch and pressuresensitive surface of tablet 130; and so on. Tablet 130 is configured todetect the various manipulations performed by the user with stylus 140and/or with a finger, knuckle, etc. on the surface of tablet 130 andcommunicate information regarding the manipulations to tablet/stylusinput module 122 on computer device 100, for example via a wired orwireless interface.

Tablet/stylus input module 122 may be implemented as a component ormodule of application 120, as a library function, as a driver, or assome other software entity. Tablet/stylus input module 122 may beimplemented in software, in hardware, or as a combination of hardwareand software. Graphics application 120, via tablet/stylus input module122, may interpret the information regarding the manipulations to detectvarious gestures and to perform various painting actions in response tothe detected gestures for creating or editing content of image 126. Forat least some of those actions, painting tool 128 may be appropriatelymoved, modified, and/or otherwise affected on display 102. Variousexamples of gestures that may be detected are listed below, as arevarious examples of painting actions that may be invoked and/orcontrolled by such stylus gestures.

In some embodiments, software and/or hardware on tablet 130 may performat least some of the functionality of detecting various gestures. Thus,in some embodiments, tablet 130 may be configured to detect gestures andcommunicate the detected gestures to graphics application 120, whichthen performs the appropriate painting actions in response to thegestures. In other embodiments, tablet 130 may only collect informationregarding gestures and communicate the gestures to application 120 viatablet/stylus input module 122; tablet/stylus input module 122 mayperform the function of detecting the gestures from the information andcommunicating the gestures to application 120, or to other modules ofapplication 120, which then performs the appropriate painting actions inresponse to the gestures.

Graphics application 120 may include a bristle-based brush model 123, inwhich the brush consists of a set of bristles that dynamically changeshape in response to the physics of the brush stroke. In contrast, asingle two-dimensional (2D) grayscale stamp is typically used byconventional digital painting programs.

Graphics application 120 (e.g., in the painting simulation module 127)may include support for “wet” and/or “dirty” paint, i.e., support forbidirectional paint transfer (e.g., from the brush to the canvas, andfrom the canvas to the brush), which enables color blending and smudgingin a way that mimics natural paint media. Bidirectional paint transferis in contrast to a unidirectional paint transfer (e.g., the transfer ofpaint from brush to canvas only, without dirtying the brush) that istypically used in conventional digital painting programs.

Graphics application 120 (e.g., in the painting simulation module 127)may simulate watercolor painting, which creates the effects of a brushwet with watery paint that slowly dries during a stroke. Conventionalpaint programs typically do not simulate these secondary effects,although some may use additional tools and textures to create similarresults.

FIG. 2 illustrates an example display 200 on which a user interface to agraphics editing module, such as image editing operations module ofgraphics application 120 may be implemented, according to oneembodiment. In this example, the display is divided into four regions orareas: menus 210, tools 202 (which may include a “fill” tool, a “clean”tool, a “tip shape” selection tool, and/or a brush type selection tool),controls 204 (which may include palette 206 and distortion parameterselection tool 205), and work area 208. Tools 202 may include one ormore user-selectable user interface elements. In this example, it isthis area that contains the user interface elements that a user mayselect to apply various effects to the image. For example, the user mayselect a type of brush tool (using the brush type selection tool) and/ora tip share (using the tip shape selection tool) for use in applyingpaint to an image being created and/or edited in work area 208. Otheroptional tools may be selected as well, such as an eraser or resetfunction, in some embodiments. While FIG. 2 shows some of the elementsin tools area 202 as buttons, other types of user interface elements,such as pop-up or pull-down menus, may be used to select from among oneor more tools in various embodiments. For example, in one embodiment,the brush type selection mechanism and/or the tip shape selectionmechanism illustrated in tools area 202 may be implemented using apop-up or pull-down menu to select a brush type (e.g., a paintbrush,pencil, charcoal, crayon, oil pastel, eraser, airbrush, spray can, andso on) and/or a tip shape (e.g., point, flat, round, square, triangle,and so on). As noted above, the reset and eraser tools are optional, andthus may or may not be included on the user interface in variousembodiments. Various embodiments may include other tools not shown aswell, such as an “undo” tool that undoes the most recent user action inthe work area 208.

In this example, controls 204 may include one or more user-modifiablecontrols, such as slider bars, dials, pop-up menus, alphanumeric textentry boxes, etc., for specifying various parameters of the paintingfunctions to be applied to an image (e.g., using the brush tool). Inthis example, two slider bars are provided to specify different values(or relative values) of configurable parameters of a painting function,one of which is usable to specify a hardness parameter value (203),e.g., for a pencil or other erodible mark making tool. In variousembodiments, slider bars may also be used to specify an amount of ink, apigment concentration amount, a transparency value, a brush width, abristle stiffness, or other parameters that are to be applied when usingthe brush tool to “paint” or “draw” on the image being created or editedin work area 208. Various methods of specifying values of any of theother parameters used in simulating painting effects (i.e., methodsother than those illustrated in FIG. 2) may be used in otherembodiments. In some embodiments, slider bars or another input mechanismin controls area 204 may be used to specify one or more thresholddistance values for use with proximity based gestures and theircorresponding functions in the graphics application, or a depositionthreshold amount. In some embodiments, slider bars or another inputmechanism in controls area 204 may be used to specify a zoom level foran automated zoom function or to override a default zoom level for sucha function.

In the example illustrated in FIG. 2, menus 206 may include one or moremenus, for example menus used to navigate to other displays in thegraphics application, open files, print or save files, undo/redoactions, and so on. In this example, work area 208 is the area in whichan image being created or edited is displayed as graphics editingoperations are performed. In various embodiments, work area 208 maydisplay a portion or all of a new image to which paint or other naturalmedia is to be added, or a portion or all of a previously existing imagebeing modified by adding paint, as described herein. In the exampleillustrated in FIG. 2, work area 208 of FIG. 2 illustrates an image inprogress.

Some embodiments of a painting simulation module, such as paintingsimulation module 127 described herein, may employ a brush model (suchas brush module 123) that simulates a brush tip and the notion of thepaint being held in the brush tip and deposited on a canvas duringstrokes. In some embodiments, during the act of stroking with the brush,the brush's paint load will be depleted, eventually running out, and thebrush may dirty, picking up paint from the canvas, as with a real brush.Clean and fill actions may be provided to allow the user to manage thepaint load between strokes for the desired stroke effect. The userinterface illustrated in FIG. 2 also includes a color palette whereby auser may manually load a brush with paint if and when desired, and a“fill” user interface element (shown as a radio button) whereby a usermay enable or disable an auto fill option.

In some embodiments, graphics application 120 may simulate theshape-changing property of a pastel, charcoal, crayon, or pencil, or anyother type of mark making tool that has an “erodible tip.” Oneembodiment of a method for creating a mark in a natural media paintingapplication using a virtual brush tool modeled with an erodible tip isillustrated in FIG. 3. The method shown in FIG. 3 may be used inconjunction with embodiments of the computer system shown in FIG. 15,among other devices. In various embodiments, some of the method elementsshown may be performed concurrently, in a different order than shown, ormay be omitted. Additional method elements may also be performed asdesired. Any of the method elements described may be performedautomatically (i.e., without user intervention). As shown, this methodmay operate as follows.

As shown in 310, a natural media painting application may model a brushtool representing an erodible media as a height map and an associatedtriangle mesh attached to a brush handle. As shown in 320, theapplication may collect or receive information about a user manipulationof a stylus representing a stroke made with the brush tool. As shown in330, the application may determine the mark to be made on the canvasdependent on the brush tool model, the collected information, and ahardness parameter of the erodible media, and the application may renderthe mark.

As shown in 340, the application may determine the change in the heightmap of the brush tool dependent on the collected information and thehardness parameter of the erodible media. As shown in 350, theapplication may store data representing an output image that includesthe virtual brush mark for display. As shown in 360, the application mayrender one or more subsequent marks with the brush tool dependent on thechanged height map.

FIG. 4 illustrates components of an example stylus 140 according to someembodiments. Stylus 140 may generally be described as having a tip 142and a handle 144. Note that a stylus 140 may be provided with two tipsinstead of one as shown in FIG. 4. Stylus 140 may include one or moreaccelerometers 148 and/or other components for sensing movement metricsincluding but not limited to spatial (location), directional, andacceleration metrics. This motion information may be communicated to atablet, such as tablet 130 shown in FIG. 1, via an interface 146.Interface 146 may typically be a wireless interface, although wiredinterfaces are also possible.

In some embodiments, the natural media painting application may employ abrush model that simulates the use of a brush, such as one representedby a brush tool that is manipulated using a stylus. In such embodiments,realistic brush behavior may be simulated in the painting application.In one embodiment, the brush behavior may be simulated substantially inreal-time to deposit ink or paint onto a virtual canvas. In someembodiments, a brush model may include a large set of discrete bristles.The bristles may comprise “virtual” bristles and may also be referred toas bristle representations. The simulated behavior of the brush mayresult in continuous strokes created by sweeping individual bristlesinto quadrilaterals. The brush model and brush behavior simulation maymodel the change of the shape of a brush tip during a stroking motionand the deposition of paint or ink resulting from the motion. In someembodiments, by computing the effect of each bristle independently ofother bristles, a faithful reproduction of how a real brush depositspaint or ink and changes shape during a stroke may be achieved.

Stylus gestures that may be detected by a natural media paintingapplication in various embodiments may include, but are not limited to:a barrel rotation (a twisting motion about the major axis of thestylus), a fanning motion (waving the stylus tip back and forth abovethe tablet), mashing down (pressing the stylus into the tablet with highpressure), a jerk up (a quick motion away from the tablet), a jerk down(a quick motion toward the tablet), shaking away from the tablet(holding the stylus by its end and flicking the wrist); an shakingtoward the tablet (holding the stylus its end and flicking the wrist).At least some of the stylus gestures that may be detected may beperformed using a stylus augmented with one or more accelerometers, andpossibly other hardware and/or software, for collecting motion and otherdata to be used in gesture recognition.

In some embodiments, a natural media painting application may detect andrecognize various stylus poses, and the pressure with which a stylustouches a tablet, and these inputs may be used to create and controlvarious painting/drawing effects, such as those described herein. Someof the stylus poses and actions that may be recognized by an interfacemodule of a natural media painting application, such as tablet/stylusinput module 122 in FIG. 1, are illustrated in FIGS. 5A-5D, according tovarious embodiments. For example, FIG. 5A illustrates a stylus which isin proximity to and being moved toward a canvas and that is being heldat an angle of less than 45°. FIG. 5B illustrates a stylus that is beingpressed into a canvas and that is being held at an angle greater than45°. 5C and 5D illustrate the effects of different stylus gestures on abrush model. For example, FIG. 5C illustrates the effect of a stylusgesture that corresponds to pressing a bristle brush lightly on acanvas, while FIG. 5D illustrates the effect of a stylus gesture thatcorresponds to mashing a bristle brush down into a canvas. In general,tablet/stylus input module 122 may be configured to recognize a widevariety of stylus poses and gestures by detecting manipulation of thestylus from an initial pose (e.g., an initial position and orientation)using six degrees of freedom (6DOF) (e.g., detecting movement in a givendirection in three dimensions, rotation about an axis in any dimension,pitch, roll, yaw, etc.) As described herein, this 6DOF information maybe augmented with information collected from an accelerometer, variousproximity sensors, a touch and/or pressure sensitive tablet device, orother input mechanisms to define a stylus gesture that is mapped to anaction to be taken in a natural media painting application, and theaction mapped to the gesture may be dependent on a work mode and/orcontext in which the stylus gesture was made. Similarly, FIGS. 6A and 6Billustrate a user holding a stylus in different poses while makingvarious painting/drawing strokes on a tablet.

As previously noted, the characteristic behaviors of artist tools suchas graphite pencils or oil pastels include interaction with canvastexture, dirtying of tip, and change in shape of tip over time due toerosion. In some embodiments, the natural media painting applicationsdescribed herein may supplement these and/or other features bysupporting the simulation of an erodible tip for various brush models.In such embodiments, when combined with full 6DOF tablet stylus input,the same range of motion employed by real-world artists using pencils,charcoal, or oil pastels, for example, may be possible. In other words,a virtual brush that supports the simulation of an erodible tip (i.e.,that mimics the same type of shape changing behavior) in a digitalpainting application may provide artists with a real artistic tool, suchthat trained artists may be able to achieve substantially the sameresults with a digital painting tool.

As previously noted, in some embodiments, the painting applicationdescribed herein may model an erodible tip as a height map and anassociated triangle mesh attached to the end of a brush handle. Theheight map resolution may be a controllable parameter determined by thebrush size and the performance limitations. While arbitrary height mapshapes may be possible, in some embodiments, the application maygenerally support brushes with a cylindrical, square, or triangle base.To create a smooth silhouette of the height map around the base, theapplication may move the vertices outside the base to the perimeter. Theapplication may then add faces connecting the edges of the height map tothe base of the brush handle to create a tip volume. In someembodiments, the application may support a variety of initial tipshapes, including, but not limited to “pointed,” “flat,” “rounded,”“square,” and “triangle” shaped tips.

FIGS. 7A-7J illustrate various embodiments of a virtual brush with anerodible tip model. For example, FIG. 7A illustrates a side view of abrush model, including a cross section of a portion of the handle and aslice of the height map at the tip. FIG. 7B illustrates an isometricview of a brush model, including the tip mesh and a portion of thehandle. In this example, the tip is flat, as illustrated by the heightmap of the tip. FIG. 7C illustrates the top view of the height map andhandle cross section. FIG. 7D illustrates an aliased silhouette of theheight map. FIG. 7E illustrates a smoothed silhouette of the height map.FIGS. 7F-7J illustrate brush models with erodible tips of differentshapes, including pointed (7F), flat (7G), rounded (7H), square (7I),and triangular (7J).

In some embodiments, to determine the instantaneous 2D stamp thatcorresponds to the contact area between the posed erodible tip and thecanvas, the application may implement the following algorithm. First,given the pose of the height map, the closest point to the canvas may bedetermined. The height at which this point is in contact with the canvasmay be mapped to pressure p=0. To simulate the effect of increasedpressure resulting in a larger contact area (due to canvas deformation),as p→1, the height may be lowered so that the closest point canpenetrate the canvas by up to some pre-determined maximum amount.

In some embodiments, a 2D stamp may be determined by the erodible tipgeometry that is below a pre-determined deposition threshold, which isdefined as a small height above the canvas. CPU triangle rasterizationwith per-pixel depth clipping may be used to create the stamp image.

In some embodiments, to create higher quality marks, the intensity ofeach pixel in the stamp may be modulated by how deeply the pixel passesthe deposition threshold. For example, a pixel that has depth right atthe deposition threshold may have intensity just above zero, while apixel that has maximum depth may have maximum intensity. However, thedepth-intensity relationship may not necessarily be linear, in differentembodiments.

FIGS. 8A-8C illustrate stamp generation based on an erodible tip contactarea, according to some embodiments. For example, FIG. 8A illustrates aportion of tip that passes below a deposition threshold (which isrepresented by the dotted line), and the resulting stamp that isgenerated based on the amount of the tip that passed below thedeposition threshold. In FIG. 8A, the solid dot at the end of the tip isthe nearest point to the canvas. FIG. 8B illustrates an erodible tip incontact with the canvas with pressure=0. FIG. 8C illustrates an erodibletip in contact with the canvas with pressure=1 (i.e., the maximumvalue).

FIG. 9 is a flow diagram illustrating one embodiment of a method forstamp generation a natural media painting application. The method shownin FIG. 9 may be used in conjunction with embodiments of the computersystem shown in FIG. 15, among other devices. In various embodiments,some of the method elements shown may be performed concurrently, in adifferent order than shown, or may be omitted. Additional methodelements may also be performed as desired. Any of the method elementsdescribed may be performed automatically (i.e., without userintervention). As shown, this method may operate as follows.

As shown in 910, for a stroke with a virtual brush tool representing anerodible media, a natural media painting application may map 6DOF datafrom the stylus, including position and orientation, to the pose of thevirtual brush tool. As shown in 920, the application may determine theclosest point of the posed erodible tip to the virtual canvas, and theapplication may map the height at which this point is in contact withthe canvas to pressure=0. As shown in 930, during the stroke, theapplication may determine the amount of the erodible tip geometry thatis below a pre-determined deposition threshold above the canvas, and theapplication may create a stamp image dependent on that geometry. Asshown in 940, the application may determine the intensity of each pixelof the mark dependent on the depth at which the pixel is below thedeposition threshold. As shown in 950, the application may render themark on the canvas dependent on the created stamp and the determinedintensity of each pixel of the stamp. As shown in 960, the applicationmay output data representing a modified image that includes the mark,e.g., for storage and/or display.

In some embodiments, the erodible tip model may also include a“hardness” parameter (whose value may be user-configurable), which maycapture the variations in behaviors from soft (8B) to hard (H8) graphitepencils, for example. In general, hard tips may have a maximum canvaspenetration that is shallower than that of soft tips, because even withhigh pressure, hard pencils may not make large marks, whereas highpressure applied to soft pencils typically does. Also, high pressure ona hard pencil may not create a very dark mark, whereas high pressure ona soft pencil typically creates a darker mark. In some embodiments,these effects may be achieved in the digital painting applicationthrough the use of the aforementioned depth-intensity mapping.

In some embodiments, a hardness parameter may also used to affect howpressure interacts with the influence of canvas texture. For example,for both soft and hard tips, low pressure strokes may be highlyinfluenced by canvas texture because the tip may just graze the crestsof the canvas without significantly deforming the canvas or the tip.Under high pressure, however, soft tips may quickly deform to fill inthe canvas valleys, while hard tips may still have varying deposition,as they tend to skip across canvas valleys. Therefore, the influence ofcanvas texture on the output stroke may be modulated by tip hardness.

One feature of an erodible tip is the actual change in tip shape overthe course of a stroke. In some embodiments, to create this behavior inthe digital painting application, at each step the mesh vertices thatare in contact within the canvas (i.e., below the deposition threshold)may be reduced in height by some amount. In some embodiments, theerosion rate may be based on data that has been empirically determined,e.g., by measuring the change in length of pencil tips after stroking atconstant pressure over a large distance. In one example, measurementsmay be made for low, medium, and high pressure, for pencils of hardnessfrom 8B to 8H, for strokes one meter long. Based on this data, a valueof erosion per distance may be computed for 9 combinations of pressure(low, medium, high) and hardness (soft, medium, hard) and stored in atable. Continuously varying values may then be interpolated from thistable. Similar tables may be constructed for other media usingmeasurements or by extrapolating from the pencil data, in differentembodiments.

In some embodiments of the digital painting application, at each step ofa stroke, the pressure and the hardness may be used to look up anerosion rate, which may be combined with the distance traveled since thelast step to compute the volume of pigment eroded. This volume may bedistributed among the vertices in contact with the canvas, which may bereduced in height correspondingly to remove the computed volume. In someembodiments, to conserve the total volume of the tip material so that itdoes not run out after many strokes, the removed volume may be addedback into the tip model at the base of the volume, e.g., keeping thetotal volume constant.

FIGS. 10A-10B illustrate the erosion of an erodible tip during a stroke,according to some embodiments. For example, FIG. 10A illustrates anerodible tip (modeled as a mesh) for which several of the vertices arebelow the canvas (or deposition threshold), prior to erosion. FIG. 10Billustrates the same tip following the erosion of the tip at thevertices that were below the canvas (or deposition threshold).

FIG. 11 is a flow diagram illustrating the use of an erodible tip in anatural media painting application, according to some embodiments. Themethod shown in FIG. 11 may be used in conjunction with embodiments ofthe computer system shown in FIG. 15, among other devices. In variousembodiments, some of the method elements shown may be performedconcurrently, in a different order than shown, or may be omitted.Additional method elements may also be performed as desired. Any of themethod elements described may be performed automatically (i.e., withoutuser intervention). As shown, this method may operate as follows.

As shown in 1110, a natural media painting application may model avirtual brush tool representing an erodible media as a height map and anassociated triangle mesh attached to a brush handle. As shown in 1120,for a stroke using the virtual brush tool, the application may collect6DOF data from the stylus, including position and orientationinformation, and the application may also collect pressure data from thetablet. As shown in 1130, at a given point in the stroke, theapplication may determine the erosion rate of the erodible tip dependenton the collected pressure data and a hardness parameter for the erodiblemedia. As shown in 1140, the application may determine the mesh verticesof the erodible tip geometry that are below a pre-determined depositionthreshold above the canvas, and the application may also determine thedistance recently traveled. As shown in 1150, the application maydetermine the volume of the tip material that has been eroded by thestroke, and the application may distribute the eroded volume between thedetermined mesh vertices, reducing the height map accordingly. As shownin 1160, in some embodiments, the application may replace the erodedvolume of tip material at the base of the volume for subsequent use.

Another aspect of the way artists use pencils or pastels is that theymay hold the tool at a shallow angle, which allows the side of the tipto be completely in contact with the canvas, creating a broad mark, suchas the type frequently used for shading. Pointed tips are most commonlyused for this. A pointed tip is generally very sharp, allowing tiltangles of up to 80°. However, some tablet styluses such as the WacomIntuos unit have maximum tilt angles in the range of 60°, with themaximum achievable tilt being more like 50° (due to the physical formfactor of the stylus). A pencil tip with a peak angle of 40° would beneeded to make it possible for an artist to maximally tilt the stylusand make the broad side of the tip contact the canvas, which would notbe very similar to real pencils.

In some embodiments, to achieve a shading effect as described above, theapplication may apply scaling to the input tilt data so that, at themaximum stylus tilt T, the broad side of the virtual erodible tip isflush with the virtual canvas, regardless of the peak angle of the tipand the tilt limitations of the tablet. In some embodiments, this may bedone by computing the maximum necessary tilt based on the tip's peakangle E, and then applying a linear scale from input tilt t to scaledtilt t′, as follows:

$\begin{matrix}{t^{\prime} = {\frac{90 - \frac{E}{2}}{T}t}} & (1)\end{matrix}$

FIGS. 12A-12D illustrate stylus tilt scaling, according to oneembodiment. For example, FIG. 12A illustrates the tip of a #2 pencil,for which the peak angle (E) is about 10°. FIG. 12B illustrates a flushpose of a pencil for making broad strokes. In FIG. 12B, thecorresponding maximum tilt θ is calculated as in equation (1) above.FIG. 12C illustrates a Wacom stylus, which has a much higher peak angle.FIG. 12D illustrates that the maximum tilt of the stylus (T) istypically about 50°. In some embodiments, the techniques describedherein may be used to map the maximum tilt of the stylus (as in 12D) tothe maximum tilt of a pencil (as in 12B).

FIG. 13 is a flow diagram illustrating one embodiment of a method forscaling the tilt of a stylus in natural media painting application. Themethod shown in FIG. 13 may be used in conjunction with embodiments ofthe computer system shown in FIG. 15, among other devices. In variousembodiments, some of the method elements shown may be performedconcurrently, in a different order than shown, or may be omitted.Additional method elements may also be performed as desired. Any of themethod elements described may be performed automatically (i.e., withoutuser intervention). As shown, this method may operate as follows.

As shown in 1310, a natural media painting application may determine themaximum tilt of the stylus. As shown in 1320, the application maydetermine the peak angle of the tip of a selected brush tool. As shownin 1330, the application may map the maximum tilt of the stylus to thetilt of the selected brush tool that is necessary to achieve the peakangle of the tip of the selected brush tool. As shown in 1340, for astroke using the selected brush tool, the application may map 6DOF datafrom the stylus, including position and orientation, to the pose of thebrush tool. As shown in 1350, the application may determine the tilt ofthe selected brush tool dependent on the tilt of the stylus and themapping.

Other digital painting applications have attempted to create asimulation of an erodible colored pencil. However, these applicationsgenerally employ a much more computationally expensive approach than theapproach described herein, such that their approach cannot be easilyintegrated into existing pixel painting pipelines. Other existingsimulations of erodible media do not directly simulate the erodible tip,do not change due to tip erosion, and/or do not support varyinghardness.

In various embodiments, the techniques described herein may be used toachieve realistic pastel and pencil simulation of high quality with lowcomputational requirements, when compared to previous approaches. Thetechniques described herein may also provide greater and more intuitivesupport for tablet stylus input, e.g., using 6DOF pose mapping and tiltscaling.

FIG. 14 illustrates components of an example tablet input deviceaccording to some embodiments. As illustrated in this example, tablet130 may include a touch and pressure-sensitive surface 832 that may beconfigured to detect contact with tip 142 of stylus 140, and/or contactwith another object such as the user's fingertip or knuckle. Surface 832may also be configured to detect motion on the surface, for exampledetecting the dragging of tip 142 of stylus 140 across the surface.Surface 832 may also be configured to detect the amount of pressureapplied to the surface, e.g., by stylus 140, another object, or a usertouch. Tablet 130 may also include an interface to stylus 836 that isconfigured to detect the position of, and motion of, stylus 140 inrelation to tablet 130, for example by receiving input from stylus 140via a wireless interface, or alternatively via one or more motiondetectors integrated in or coupled to tablet 130 that are configured totrack the motion and position of stylus 140. In some embodiments, tablet130 and/or stylus 140 may include a camera, through which input aboutthe position and/or motion of stylus 140 may be collected (not shown),or such a camera may included as an additional component of the systemseparate from tablet 130 and stylus 140. In some embodiments, tablet 130may also include an input processing module 838 configured to processinput received via interface to stylus 836 and/or surface 832.

Input processing module 838 may also include an interface to computerdevice 834. Interface 834 may be a wired or wireless interface.Interface 834 may be configured to communicate information collectedfrom interface 836 and/or surface 832 to a computer device such ascomputer device 100 of FIG. 1. A graphics application on the computerdevice, such as graphics application 120 of FIG. 1, may interpret theinformation to detect various gestures and to perform various paintingactions in response to the detected gestures for creating or editing thecontent of images, as described herein. In some embodiments, inputprocessing module 838 may be configured to perform at least some of thefunctionality of detecting and/or recognizing various gestures. Thus, insome embodiments, tablet 130 may be configured to detect/recognizegestures and communicate the gestures to a graphics application viainterface 834. The graphics application may then perform the appropriatepainting actions in response to the gestures.

Some embodiments may include a means for detecting poses and gesturesmade using a stylus, a tablet type input device, and/or a combination ofa stylus and a tablet type input device. For example, a tablet/stylusinput module may present an interface through which various poses orgestures representing actions to be taken or painting effects to beapplied in a natural media painting application (e.g., mode changesand/or painting operations) may be detected (e.g., using collectedmotion information, pressure data, etc.) and recognized, and maygenerate and store data representing the detected poses or gestures foruse in various image editing operations in the natural media paintingapplication, as described herein. The tablet/stylus input module may insome embodiments be implemented by a non-transitory, computer-readablestorage medium and one or more processors (e.g., CPUs and/or GPUs) of acomputing apparatus. The computer-readable storage medium may storeprogram instructions executable by the one or more processors to causethe computing apparatus to perform presenting an interface through whichvarious poses or gestures may be detected and recognized, detecting andrecognizing those poses or gestures, and generating and storing datarepresenting those poses or gestures for subsequent use in the naturalmedia painting application, as described herein. Other embodiments ofthe tablet/stylus input module may be at least partially implemented byhardware circuitry and/or firmware stored, for example, in anon-volatile memory.

Some embodiments may include a means for mapping detected poses and/orgestures made using a stylus and/or tablet type input device to variousfunctions of a natural media painting application. For example, apose/gesture mapping module may receive input specifying various posesor gestures that have been detected, may determine actions to be takenin a natural media painting application (e.g., image editing operationsto be performed using a brush tool in the application, orpainting/drawing effects to be applied in the application) in responseto that input, and may generate and store data representing the actionsto be taken or the effects to be applied in the natural media paintingapplication, as described herein. The pose/gesture mapping module may insome embodiments be implemented by a non-transitory, computer-readablestorage medium and one or more processors (e.g., CPUs and/or GPUs) of acomputing apparatus. The computer-readable storage medium may storeprogram instructions executable by the one or more processors to causethe computing apparatus to perform receiving input specifying variousstylus poses or gestures that have been detected, determining actions tobe taken or effects to be applied in a natural media paintingapplication in response to that input, and generating and storing datarepresenting the actions to be taken or the effects to be applied in thenatural media painting application, as described herein. Otherembodiments of the pose/gesture mapping module may be at least partiallyimplemented by hardware circuitry and/or firmware stored, for example,in a non-volatile memory.

Some embodiments may include a means for simulating the behavior ofvarious types of brushes in a natural media painting application. Forexample, a painting simulation module (which may include a brush model),may receive input specifying various painting actions to be performed ina natural media painting application (e.g., image editing operations tobe performed using a brush tool in the application) in response totablet and/or stylus input, and may generate and store data representingan image that has been modified by the various image editing operationsin the natural media painting application, as described herein. Thepainting simulation module may in some embodiments be implemented by anon-transitory, computer-readable storage medium and one or moreprocessors (e.g., CPUs and/or GPUs) of a computing apparatus. Thecomputer-readable storage medium may store program instructionsexecutable by the one or more processors to cause the computingapparatus to perform receiving input specifying various painting actionsto be performed in a natural media painting application (e.g., imageediting operations to be performed using a brush tool in theapplication) in response to tablet and/or stylus input, and generatingand storing data representing an image that has been modified by thevarious image editing operations in the natural media paintingapplication, as described herein. Other embodiments of the paintingsimulation module may be at least partially implemented by hardwarecircuitry and/or firmware stored, for example, in a non-volatile memory.

Example Computer System

The methods illustrated and described herein may be executed on one ormore computer systems, which may interact with other devices, accordingto various embodiments. One such computer system is illustrated in FIG.15. In the illustrated embodiment, computer system 1900 includes one ormore processors 1910 coupled to a system memory 1920 via an input/output(I/O) interface 1930. Computer system 1900 further includes a networkinterface 1940 coupled to I/O interface 1930, and one or moreinput/output devices 1950, such as cursor control device 1960, keyboard1970, audio device 1990, and display(s) 1980. Input/output devices 1950include a tablet 130 and stylus 140 for enabling natural media paintingusing a realistic brush and tablet stylus gestures as described herein.In some embodiments, it is contemplated that embodiments may beimplemented using a single instance of computer system 1900, while inother embodiments multiple such systems, or multiple nodes making upcomputer system 1900, may be configured to host different portions orinstances of embodiments. For example, in one embodiment some elementsmay be implemented via one or more nodes of computer system 1900 thatare distinct from those nodes implementing other elements.

In various embodiments, computer system 1900 may be a uniprocessorsystem including one processor 1910 or a multiprocessor system includingseveral processors 1910 (e.g., two, four, eight, or another suitablenumber). Processor(s) 1910 may be any suitable processor capable ofexecuting instructions. For example, in various embodiments, processors1910 may be general-purpose or embedded processors implementing any of avariety of instruction set architectures (ISAs), such as the x86,PowerPC, SPARC, or MIPS ISAs, or any other suitable ISA. Inmultiprocessor systems, each of processors 1910 may commonly, but notnecessarily, implement the same ISA.

In some embodiments, at least one processor 1910 may be a graphicsprocessing unit. A graphics processing unit or GPU may be considered adedicated graphics-rendering device for a personal computer,workstation, game console or other computer system. Modern GPUs may bevery efficient at manipulating and displaying computer graphics, andtheir highly parallel structure may make them more effective thantypical CPUs for a range of complex graphical algorithms. For example, agraphics processor may implement a number of graphics primitiveoperations in a way that makes executing them much faster than drawingdirectly to the screen with a host central processing unit (CPU). Invarious embodiments, the methods as illustrated and described in theaccompanying description may be implemented by program instructionsconfigured for execution on one of, or parallel execution on two or moreof, such GPUs. The GPU(s) may implement one or more applicationprogrammer interfaces (APIs) that permit programmers to invoke thefunctionality of the GPU(s). Suitable GPUs may be commercially availablefrom vendors such as NVIDIA Corporation, ATI Technologies, and others.

System memory 1920 may be configured to store program instructionsand/or data accessible by processor 1910. In various embodiments, systemmemory 1920 may be implemented using any suitable memory technology,such as static random access memory (SRAM), synchronous dynamic RAM(SDRAM), nonvolatile/Flash-type memory, or any other type of memory. Inthe illustrated embodiment, program instructions and data implementingdesired functions, such as those for methods as illustrated anddescribed in the accompanying description, are shown stored withinsystem memory 1920 as program instructions 1925 and data storage 1935,respectively. In other embodiments, program instructions and/or data maybe received, sent or stored upon different types of computer-accessiblemedia or on similar media separate from system memory 1920 or computersystem 1900. Generally speaking, a computer-accessible medium mayinclude storage media or memory media such as magnetic or optical media,e.g., disk or CD/DVD-ROM coupled to computer system 1900 via I/Ointerface 1930. Program instructions and data stored via acomputer-accessible medium may be transmitted by transmission media orsignals such as electrical, electromagnetic, or digital signals, whichmay be conveyed via a communication medium such as a network and/or awireless link, such as may be implemented via network interface 1940.

In one embodiment, I/O interface 1930 may be configured to coordinateI/O traffic between processor 1910, system memory 1920, and anyperipheral devices in the device, including network interface 1940 orother peripheral interfaces, such as input/output devices 1950,including tablet 130 and stylus 140. In some embodiments, I/O interface1930 may perform any necessary protocol, timing or other datatransformations to convert data signals from one component (e.g., systemmemory 1920) into a format suitable for use by another component (e.g.,processor 1910). In some embodiments, I/O interface 1930 may includesupport for devices attached through various types of peripheral buses,such as a variant of the Peripheral Component Interconnect (PCI) busstandard or the Universal Serial Bus (USB) standard, for example. Insome embodiments, the function of I/O interface 1930 may be split intotwo or more separate components, such as a north bridge and a southbridge, for example. In addition, in some embodiments some or all of thefunctionality of I/O interface 1930, such as an interface to systemmemory 1920, may be incorporated directly into processor 1910.

Network interface 1940 may be configured to allow data to be exchangedbetween computer system 1900 and other devices attached to a network,such as other computer systems, or between nodes of computer system1900. In various embodiments, network interface 1940 may supportcommunication via wired or wireless general data networks, such as anysuitable type of Ethernet network, for example; viatelecommunications/telephony networks such as analog voice networks ordigital fiber communications networks; via storage area networks such asFibre Channel SANs, or via any other suitable type of network and/orprotocol.

Input/output devices 1950 may, in some embodiments, include one or moredisplay terminals, keyboards, keypads, touchpads, scanning devices,voice or optical recognition devices, tablets and stylus, or any otherdevices suitable for entering or retrieving data by one or more computersystem 1900. Multiple input/output devices 1950 may be present incomputer system 1900 or may be distributed on various nodes of computersystem 1900. In some embodiments, similar input/output devices may beseparate from computer system 1900 and may interact with one or morenodes of computer system 1900 through a wired or wireless connection,such as over network interface 1940.

As shown in FIG. 15, memory 1920 may include program instructions 1925,configured to implement embodiments of methods as illustrated anddescribed in the accompanying description, and data storage 1935,comprising various data accessible by program instructions 1925. In oneembodiment, program instructions 1925 may include software elements ofmethods as illustrated and described in the accompanying description,including a tablet/stylus input module, painting simulation module,brush model, and/or pose/gesture mapping module. Data storage 1935 mayinclude data that may be used by these and other modules in someembodiments. In other embodiments, other or different software elementsand/or data may be included in memory 1920.

Those skilled in the art will appreciate that computer system 1900 ismerely illustrative and is not intended to limit the scope of methods asillustrated and described in the accompanying description. Inparticular, the computer system and devices may include any combinationof hardware or software that can perform the indicated functions,including computers, network devices, internet appliances, PDAs,wireless phones, pagers, etc. Computer system 1900 may also be connectedto other devices that are not illustrated, or instead may operate as astand-alone system. In addition, the functionality provided by theillustrated components may in some embodiments be combined in fewercomponents or distributed in additional components. Similarly, in someembodiments, the functionality of some of the illustrated components maynot be provided and/or other additional functionality may be available.

Those skilled in the art will also appreciate that, while various itemsare illustrated as being stored in memory or on storage while beingused, these items or portions of them may be transferred between memoryand other storage devices for purposes of memory management and dataintegrity. Alternatively, in other embodiments some or all of thesoftware components may execute in memory on another device andcommunicate with the illustrated computer system via inter-computercommunication. Some or all of the system components or data structuresmay also be stored (e.g., as instructions or structured data) on acomputer-accessible medium or a portable article to be read by anappropriate drive, various examples of which are described above. Insome embodiments, instructions stored on a computer-accessible mediumseparate from computer system 1900 may be transmitted to computer system1900 via transmission media or signals such as electrical,electromagnetic, or digital signals, conveyed via a communication mediumsuch as a network and/or a wireless link. Various embodiments mayfurther include receiving, sending or storing instructions and/or dataimplemented in accordance with the foregoing description upon acomputer-accessible medium. Accordingly, the systems and methodsdescribed herein may be practiced with other computer systemconfigurations.

Various embodiments may further include receiving, sending or storinginstructions and/or data implemented in accordance with the foregoingdescription upon a computer-accessible medium. Generally speaking, acomputer-accessible medium may include storage media or memory mediasuch as magnetic or optical media, e.g., disk or DVD/CD-ROM, volatile ornon-volatile media such as RAM (e.g. SDRAM, DDR, RDRAM, SRAM, etc.),ROM, etc., as well as transmission media or signals such as electrical,electromagnetic, or digital signals, conveyed via a communication mediumsuch as network and/or a wireless link.

The various methods as illustrated in the figures and described hereinrepresent examples of embodiments of methods. The methods may beimplemented in software, hardware, or a combination thereof. The orderof method may be changed, and various elements may be added, reordered,combined, omitted, modified, etc. Various modifications and changes maybe made as would be obvious to a person skilled in the art having thebenefit of this disclosure. It is intended that the disclosure embraceall such modifications and changes and, accordingly, the abovedescription to be regarded in an illustrative rather than a restrictivesense.

1-20. (canceled)
 21. A computer-implemented method to model a brush toolas an erodible input media, the method comprising: modeling the brushtool as the erodible input media based at least in part on a stiffnessparameter that represents a stiffness of the brush tool; determining abrush stroke on a virtual canvas dependent on the brush tool model andthe stiffness parameter; determining a change in the brush tool modeldue to the brush stroke on the virtual canvas and the stiffnessparameter.
 22. The computer-implemented method as recited in claim 21,wherein a stiffness parameter value of the stiffness parameter varies abehavior of the brush tool based at least in part on an amount ofpressure that interacts with an influence of a texture of the virtualcanvas.
 23. The computer-implemented method as recited in claim 21,wherein said determining the brush stroke on the virtual canvas isfurther dependent on a texture of the virtual canvas and pressureapplied to create the brush stroke.
 24. The computer-implemented methodas recited in claim 21, further comprising: representing the brush toolas a height map; and determining a change in the height map due to thebrush stroke on the virtual canvas and the stiffness parameter.
 25. Thecomputer-implemented method as recited in claim 24, further comprisingrendering a subsequent brush stroke on the virtual canvas based on thedetermined change in the height map that represents the brush tool. 26.The computer-implemented method as recited in claim 24, wherein saiddetermining the change in the height map is further due to pressureapplied to create the brush stroke on the virtual canvas.
 27. Thecomputer-implemented method as recited in claim 21, further comprisinggenerating a natural media painting simulation based on the modeledbrush tool and a texture of the virtual canvas.
 28. Thecomputer-implemented method as recited in claim 21, wherein thestiffness parameter is user-configurable, effective to control how atexture of the virtual canvas and a pressure applied to create the brushstroke on the virtual canvas influence said modeling the brush tool. 29.The computer-implemented method as recited in claim 21, furthercomprising modeling an erosion rate of the brush tool to determine achange in shape of the brush tool.
 30. A system, comprising: a memoryconfigured to maintain a model of an input tool as an erodible inputmedia; at least one processor to implement a graphics application thatis configured to: model the input tool as the erodible input media basedat least in part on a hardness parameter that represents a hardness ofthe input tool; determine an input tool mark on a virtual canvasdependent on the input tool model and the hardness parameter; anddetermine a change in the input tool model due to the input tool mark onthe virtual canvas and the hardness parameter.
 31. The system as recitedin claim 30, wherein a hardness parameter value of the hardnessparameter varies a behavior of the input tool based at least in part onan amount of pressure that interacts with an influence of a texture ofthe virtual canvas.
 32. The system as recited in claim 30, wherein thegraphics application is configured to said determine the input tool markon the virtual canvas further dependent on a texture of the virtualcanvas and pressure applied to create the input tool mark.
 33. Thesystem as recited in claim 30, wherein the graphics application isconfigured to: represent the input tool as a height map; determine achange in the height map due to the input tool mark on the virtualcanvas and the hardness parameter; and render a subsequent input toolmark on the virtual canvas based on the determined change in the heightmap that represents the input tool.
 34. The system as recited in claim33, wherein the graphics application is configured to said determine thechange in the height map further due to pressure applied to create theinput tool mark on the virtual canvas.
 35. The system as recited inclaim 30, wherein the graphics application is configured to generate anatural media sketch simulation based on the modeled input tool and atexture of the virtual canvas, the input tool representing one of acharcoal stick, a crayon, or a pencil having an erodible tip.
 36. Thesystem as recited in claim 30, wherein the hardness parameter isuser-configurable, effective to control how a texture of the virtualcanvas and a pressure applied to create the input tool mark on thevirtual canvas influence the model of the brush tool.
 37. The system asrecited in claim 30, wherein the graphics application is configured tomodel an erosion rate of the input tool to determine a change in shapeof an erodible tip of the input tool.
 38. A computer-implemented methodto model an input tool as an erodible input media, the methodcomprising: modeling the input tool as the erodible input media based atleast in part on a hardness parameter that represents a hardness of anerodible tip of the input tool; determine an input tool mark on avirtual canvas dependent on the input tool model and the hardnessparameter; and determine a change in the input tool model due to theinput tool mark on the virtual canvas and the hardness parameter. 39.The computer-implemented method as recited in claim 38, wherein saiddetermining the input tool mark on the virtual canvas is furtherdependent on a texture of the virtual canvas and pressure applied tocreate the input too mark.
 40. The computer-implemented method asrecited in claim 38, further comprising generating a natural mediasketch simulation based on the modeled input tool and a texture of thevirtual canvas, the input tool representing one of a charcoal stick, acrayon, or a pencil having the erodible tip.